{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lora 实战"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step1 导入相关包"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\Miniconda\\envs\\geo\\lib\\site-packages\\numpy\\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:\n",
      "d:\\Miniconda\\envs\\geo\\lib\\site-packages\\numpy\\.libs\\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll\n",
      "d:\\Miniconda\\envs\\geo\\lib\\site-packages\\numpy\\.libs\\libopenblas64__v0.3.21-gcc_10_3_0.dll\n",
      "  warnings.warn(\"loaded more than 1 DLL from .libs:\"\n"
     ]
    }
   ],
   "source": [
    "from datasets import Dataset,load_dataset\n",
    "from transformers import AutoTokenizer, AutoModelForCausalLM, DataCollatorForSeq2Seq, TrainingArguments, Trainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dir = '/data/datasets/alpaca_data_zh/alpaca_gpt4_data_zh.json'\n",
    "pretrain_model_dir = \"/data/models/huggingface/bloom-1b4-zh\"\n",
    "save_dir = '/data/logs/conversation_rebot_for_1b4_lora_tuning'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step2 加载数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset({\n",
      "    features: ['instruction', 'output', 'input'],\n",
      "    num_rows: 9763\n",
      "})\n"
     ]
    }
   ],
   "source": [
    "datasets = load_dataset('json', data_files=data_dir,split='train')\n",
    "datasets = datasets.train_test_split(test_size=0.8)\n",
    "ds = datasets[\"train\"]\n",
    "print(ds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'instruction': ['利用人工智能在法律领域的两个好处。',\n",
       "  '写一个关于一个人记得了他应该忘记的事情的情境故事。',\n",
       "  '撰写一篇关于情感在决策中的重要性的文章。'],\n",
       " 'output': ['人工智能在法律领域的两个好处包括：\\n\\n1. 提高效率和准确性：人工智能能够通过自然语言处理和机器学习技术来处理大量的法律文书资料，从而能够快速找出相关的案例，法律条文和先例来支持案件。这不仅能提高律师的工作效率，还能减少人为错误和遗漏，提高法律服务的质量和准确性。\\n\\n2. 降低法律服务成本：人工智能可以对简单的法律问题进行自动化处理，例如起草合同，撰写法律文书等。这可以大大降低法律服务的成本，使普通民众能够更容易获得法律帮助。同时，人工智能也能帮助律师进行案件筛选，精简工作流程，降低人力成本，提高律所的盈利能力。',\n",
       "  '李明一觉醒来，突然想起十年前的一件事。那本应该是一件被他深深埋藏、不愿提及的往事。他当时在公司里犯了一个严重的错误，导致公司损失惨重。在那时，这件事情曾对李明产生过极大的压力，致使他一度想辞职离开。最终，他决定挑战自我，努力弥补过失。花费了几年的时间，他才慢慢挽回了信任，恢复了原有的地位。那段经历曾经令他痛不欲生，他一直试图忘记。\\n\\n然而，今天这个记忆突然卷土重来。李明觉得有些困惑，为什么过去已久的事情还会如此清晰地浮现在他的脑海里呢？随着一天的深入，李明开始思考这段经历给他带来的影响。他发现虽然痛苦，但那真的是一次人生的重要转折点。从那以后，他变得愈加谨慎和负责任，因此在工作上也取得了更大的成就。\\n\\n最终，李明认识到，有些事情并不是应该被遗忘的，它们会成为我们成长道路上的指路明灯。他从那次失败中吸取了教训，也为自己赢得了更多的机会。他开始感谢自己记起了这段经历，并将其视为一份宝贵的财富。\\n\\n从此以后，每当李明',\n",
       "  '当人们进行决策时，情绪常常在其中扮演着重要的角色。在一些情况下，情绪可能会帮助我们快速做出决定，而在其他情况下，情绪可能会干扰我们的判断，导致我们做出错误的决定。因此，了解情绪在决策中的重要性并将其管理得当，对于我们做出明智的决策至关重要。\\n\\n首先，情绪有助于我们快速做出决策。当我们面临紧急情况或危险时，我们的情绪往往会指引我们做出快速反应。例如，当我们看到一辆快速接近的汽车时，我们的恐惧情绪会促使我们尽快离开道路。在这种情况下，情绪能够帮助我们快速做出保护自己的决定。\\n\\n其次，情绪能够帮助我们建立情感联系，从而为我们的决策提供更多信息。当我们在做出决策时，我们需要考虑到我们周围的人和我们与他们的关系。情绪能够帮助我们了解我们与他人的关系，以及我们对他们的感觉。这些信息对于我们做出明智的决定至关重要。\\n\\n然而，情绪也可能阻碍我们做出明智的决定。当我们被愤怒、悲伤、压抑或其他强烈情绪所左右时，我们的判断力可能会受到影响。在这种情况下，我们往往容易做出冲动的决定，而不是'],\n",
       " 'input': ['', '', '']}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step3 数据集预处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BloomTokenizerFast(name_or_path='/data/models/huggingface/bloom-1b4-zh', vocab_size=46145, model_max_length=1000000000000000019884624838656, is_fast=True, padding_side='left', truncation_side='right', special_tokens={'bos_token': '<s>', 'eos_token': '</s>', 'unk_token': '<unk>', 'pad_token': '<pad>'}, clean_up_tokenization_spaces=False),  added_tokens_decoder={\n",
       "\t0: AddedToken(\"<unk>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
       "\t1: AddedToken(\"<s>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
       "\t2: AddedToken(\"</s>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
       "\t3: AddedToken(\"<pad>\", rstrip=False, lstrip=False, single_word=False, normalized=False, special=True),\n",
       "}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained(pretrain_model_dir)\n",
    "tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def process_func(example):\n",
    "    MAX_LENGTH = 256\n",
    "    input_ids, attention_mask, labels = [], [], []\n",
    "    instruction = tokenizer(\"\\n\".join([\"Human: \" + example[\"instruction\"], example[\"input\"]]).strip() + \"\\n\\nAssistant: \")\n",
    "    response = tokenizer(example[\"output\"] + tokenizer.eos_token)\n",
    "    input_ids = instruction[\"input_ids\"] + response[\"input_ids\"]\n",
    "    attention_mask = instruction[\"attention_mask\"] + response[\"attention_mask\"]\n",
    "    labels = [-100] * len(instruction[\"input_ids\"]) + response[\"input_ids\"]\n",
    "    if len(input_ids) > MAX_LENGTH:\n",
    "        input_ids = input_ids[:MAX_LENGTH]\n",
    "        attention_mask = attention_mask[:MAX_LENGTH]\n",
    "        labels = labels[:MAX_LENGTH]\n",
    "    return {\n",
    "        \"input_ids\": input_ids,\n",
    "        \"attention_mask\": attention_mask,\n",
    "        \"labels\": labels\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f3e7b83fcc104459a5903e00508f83b5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/9763 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['input_ids', 'attention_mask', 'labels'],\n",
       "    num_rows: 9763\n",
       "})"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_ds = ds.map(process_func, remove_columns=ds.column_names)\n",
    "tokenized_ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Human: 写一个关于一个人记得了他应该忘记的事情的情境故事。\\n\\nAssistant: 李明一觉醒来，突然想起十年前的一件事。那本应该是一件被他深深埋藏、不愿提及的往事。他当时在公司里犯了一个严重的错误，导致公司损失惨重。在那时，这件事情曾对李明产生过极大的压力，致使他一度想辞职离开。最终，他决定挑战自我，努力弥补过失。花费了几年的时间，他才慢慢挽回了信任，恢复了原有的地位。那段经历曾经令他痛不欲生，他一直试图忘记。\\n\\n然而，今天这个记忆突然卷土重来。李明觉得有些困惑，为什么过去已久的事情还会如此清晰地浮现在他的脑海里呢？随着一天的深入，李明开始思考这段经历给他带来的影响。他发现虽然痛苦，但那真的是一次人生的重要转折点。从那以后，他变得愈加谨慎和负责任，因此在工作上也取得了更大的成就。\\n\\n最终，李明认识到，有些事情并不是应该被遗忘的，它们会成为我们成长道路上的指路明灯。他从那次失败中吸取了教训，也为自己赢得了更多的机会。他开始感谢自己记起了这段'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.decode(tokenized_ds[1][\"input_ids\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'李明一觉醒来，突然想起十年前的一件事。那本应该是一件被他深深埋藏、不愿提及的往事。他当时在公司里犯了一个严重的错误，导致公司损失惨重。在那时，这件事情曾对李明产生过极大的压力，致使他一度想辞职离开。最终，他决定挑战自我，努力弥补过失。花费了几年的时间，他才慢慢挽回了信任，恢复了原有的地位。那段经历曾经令他痛不欲生，他一直试图忘记。\\n\\n然而，今天这个记忆突然卷土重来。李明觉得有些困惑，为什么过去已久的事情还会如此清晰地浮现在他的脑海里呢？随着一天的深入，李明开始思考这段经历给他带来的影响。他发现虽然痛苦，但那真的是一次人生的重要转折点。从那以后，他变得愈加谨慎和负责任，因此在工作上也取得了更大的成就。\\n\\n最终，李明认识到，有些事情并不是应该被遗忘的，它们会成为我们成长道路上的指路明灯。他从那次失败中吸取了教训，也为自己赢得了更多的机会。他开始感谢自己记起了这段'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.decode(list(filter(lambda x: x != -100, tokenized_ds[1][\"labels\"])))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step4 创建模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = AutoModelForCausalLM.from_pretrained(pretrain_model_dir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "transformer.word_embeddings.weight\n",
      "transformer.word_embeddings_layernorm.weight\n",
      "transformer.word_embeddings_layernorm.bias\n",
      "transformer.h.0.input_layernorm.weight\n",
      "transformer.h.0.input_layernorm.bias\n",
      "transformer.h.0.self_attention.query_key_value.weight\n",
      "transformer.h.0.self_attention.query_key_value.bias\n",
      "transformer.h.0.self_attention.dense.weight\n",
      "transformer.h.0.self_attention.dense.bias\n",
      "transformer.h.0.post_attention_layernorm.weight\n",
      "transformer.h.0.post_attention_layernorm.bias\n",
      "transformer.h.0.mlp.dense_h_to_4h.weight\n",
      "transformer.h.0.mlp.dense_h_to_4h.bias\n",
      "transformer.h.0.mlp.dense_4h_to_h.weight\n",
      "transformer.h.0.mlp.dense_4h_to_h.bias\n",
      "transformer.h.1.input_layernorm.weight\n",
      "transformer.h.1.input_layernorm.bias\n",
      "transformer.h.1.self_attention.query_key_value.weight\n",
      "transformer.h.1.self_attention.query_key_value.bias\n",
      "transformer.h.1.self_attention.dense.weight\n",
      "transformer.h.1.self_attention.dense.bias\n",
      "transformer.h.1.post_attention_layernorm.weight\n",
      "transformer.h.1.post_attention_layernorm.bias\n",
      "transformer.h.1.mlp.dense_h_to_4h.weight\n",
      "transformer.h.1.mlp.dense_h_to_4h.bias\n",
      "transformer.h.1.mlp.dense_4h_to_h.weight\n",
      "transformer.h.1.mlp.dense_4h_to_h.bias\n",
      "transformer.h.2.input_layernorm.weight\n",
      "transformer.h.2.input_layernorm.bias\n",
      "transformer.h.2.self_attention.query_key_value.weight\n",
      "transformer.h.2.self_attention.query_key_value.bias\n",
      "transformer.h.2.self_attention.dense.weight\n",
      "transformer.h.2.self_attention.dense.bias\n",
      "transformer.h.2.post_attention_layernorm.weight\n",
      "transformer.h.2.post_attention_layernorm.bias\n",
      "transformer.h.2.mlp.dense_h_to_4h.weight\n",
      "transformer.h.2.mlp.dense_h_to_4h.bias\n",
      "transformer.h.2.mlp.dense_4h_to_h.weight\n",
      "transformer.h.2.mlp.dense_4h_to_h.bias\n",
      "transformer.h.3.input_layernorm.weight\n",
      "transformer.h.3.input_layernorm.bias\n",
      "transformer.h.3.self_attention.query_key_value.weight\n",
      "transformer.h.3.self_attention.query_key_value.bias\n",
      "transformer.h.3.self_attention.dense.weight\n",
      "transformer.h.3.self_attention.dense.bias\n",
      "transformer.h.3.post_attention_layernorm.weight\n",
      "transformer.h.3.post_attention_layernorm.bias\n",
      "transformer.h.3.mlp.dense_h_to_4h.weight\n",
      "transformer.h.3.mlp.dense_h_to_4h.bias\n",
      "transformer.h.3.mlp.dense_4h_to_h.weight\n",
      "transformer.h.3.mlp.dense_4h_to_h.bias\n",
      "transformer.h.4.input_layernorm.weight\n",
      "transformer.h.4.input_layernorm.bias\n",
      "transformer.h.4.self_attention.query_key_value.weight\n",
      "transformer.h.4.self_attention.query_key_value.bias\n",
      "transformer.h.4.self_attention.dense.weight\n",
      "transformer.h.4.self_attention.dense.bias\n",
      "transformer.h.4.post_attention_layernorm.weight\n",
      "transformer.h.4.post_attention_layernorm.bias\n",
      "transformer.h.4.mlp.dense_h_to_4h.weight\n",
      "transformer.h.4.mlp.dense_h_to_4h.bias\n",
      "transformer.h.4.mlp.dense_4h_to_h.weight\n",
      "transformer.h.4.mlp.dense_4h_to_h.bias\n",
      "transformer.h.5.input_layernorm.weight\n",
      "transformer.h.5.input_layernorm.bias\n",
      "transformer.h.5.self_attention.query_key_value.weight\n",
      "transformer.h.5.self_attention.query_key_value.bias\n",
      "transformer.h.5.self_attention.dense.weight\n",
      "transformer.h.5.self_attention.dense.bias\n",
      "transformer.h.5.post_attention_layernorm.weight\n",
      "transformer.h.5.post_attention_layernorm.bias\n",
      "transformer.h.5.mlp.dense_h_to_4h.weight\n",
      "transformer.h.5.mlp.dense_h_to_4h.bias\n",
      "transformer.h.5.mlp.dense_4h_to_h.weight\n",
      "transformer.h.5.mlp.dense_4h_to_h.bias\n",
      "transformer.h.6.input_layernorm.weight\n",
      "transformer.h.6.input_layernorm.bias\n",
      "transformer.h.6.self_attention.query_key_value.weight\n",
      "transformer.h.6.self_attention.query_key_value.bias\n",
      "transformer.h.6.self_attention.dense.weight\n",
      "transformer.h.6.self_attention.dense.bias\n",
      "transformer.h.6.post_attention_layernorm.weight\n",
      "transformer.h.6.post_attention_layernorm.bias\n",
      "transformer.h.6.mlp.dense_h_to_4h.weight\n",
      "transformer.h.6.mlp.dense_h_to_4h.bias\n",
      "transformer.h.6.mlp.dense_4h_to_h.weight\n",
      "transformer.h.6.mlp.dense_4h_to_h.bias\n",
      "transformer.h.7.input_layernorm.weight\n",
      "transformer.h.7.input_layernorm.bias\n",
      "transformer.h.7.self_attention.query_key_value.weight\n",
      "transformer.h.7.self_attention.query_key_value.bias\n",
      "transformer.h.7.self_attention.dense.weight\n",
      "transformer.h.7.self_attention.dense.bias\n",
      "transformer.h.7.post_attention_layernorm.weight\n",
      "transformer.h.7.post_attention_layernorm.bias\n",
      "transformer.h.7.mlp.dense_h_to_4h.weight\n",
      "transformer.h.7.mlp.dense_h_to_4h.bias\n",
      "transformer.h.7.mlp.dense_4h_to_h.weight\n",
      "transformer.h.7.mlp.dense_4h_to_h.bias\n",
      "transformer.h.8.input_layernorm.weight\n",
      "transformer.h.8.input_layernorm.bias\n",
      "transformer.h.8.self_attention.query_key_value.weight\n",
      "transformer.h.8.self_attention.query_key_value.bias\n",
      "transformer.h.8.self_attention.dense.weight\n",
      "transformer.h.8.self_attention.dense.bias\n",
      "transformer.h.8.post_attention_layernorm.weight\n",
      "transformer.h.8.post_attention_layernorm.bias\n",
      "transformer.h.8.mlp.dense_h_to_4h.weight\n",
      "transformer.h.8.mlp.dense_h_to_4h.bias\n",
      "transformer.h.8.mlp.dense_4h_to_h.weight\n",
      "transformer.h.8.mlp.dense_4h_to_h.bias\n",
      "transformer.h.9.input_layernorm.weight\n",
      "transformer.h.9.input_layernorm.bias\n",
      "transformer.h.9.self_attention.query_key_value.weight\n",
      "transformer.h.9.self_attention.query_key_value.bias\n",
      "transformer.h.9.self_attention.dense.weight\n",
      "transformer.h.9.self_attention.dense.bias\n",
      "transformer.h.9.post_attention_layernorm.weight\n",
      "transformer.h.9.post_attention_layernorm.bias\n",
      "transformer.h.9.mlp.dense_h_to_4h.weight\n",
      "transformer.h.9.mlp.dense_h_to_4h.bias\n",
      "transformer.h.9.mlp.dense_4h_to_h.weight\n",
      "transformer.h.9.mlp.dense_4h_to_h.bias\n",
      "transformer.h.10.input_layernorm.weight\n",
      "transformer.h.10.input_layernorm.bias\n",
      "transformer.h.10.self_attention.query_key_value.weight\n",
      "transformer.h.10.self_attention.query_key_value.bias\n",
      "transformer.h.10.self_attention.dense.weight\n",
      "transformer.h.10.self_attention.dense.bias\n",
      "transformer.h.10.post_attention_layernorm.weight\n",
      "transformer.h.10.post_attention_layernorm.bias\n",
      "transformer.h.10.mlp.dense_h_to_4h.weight\n",
      "transformer.h.10.mlp.dense_h_to_4h.bias\n",
      "transformer.h.10.mlp.dense_4h_to_h.weight\n",
      "transformer.h.10.mlp.dense_4h_to_h.bias\n",
      "transformer.h.11.input_layernorm.weight\n",
      "transformer.h.11.input_layernorm.bias\n",
      "transformer.h.11.self_attention.query_key_value.weight\n",
      "transformer.h.11.self_attention.query_key_value.bias\n",
      "transformer.h.11.self_attention.dense.weight\n",
      "transformer.h.11.self_attention.dense.bias\n",
      "transformer.h.11.post_attention_layernorm.weight\n",
      "transformer.h.11.post_attention_layernorm.bias\n",
      "transformer.h.11.mlp.dense_h_to_4h.weight\n",
      "transformer.h.11.mlp.dense_h_to_4h.bias\n",
      "transformer.h.11.mlp.dense_4h_to_h.weight\n",
      "transformer.h.11.mlp.dense_4h_to_h.bias\n",
      "transformer.h.12.input_layernorm.weight\n",
      "transformer.h.12.input_layernorm.bias\n",
      "transformer.h.12.self_attention.query_key_value.weight\n",
      "transformer.h.12.self_attention.query_key_value.bias\n",
      "transformer.h.12.self_attention.dense.weight\n",
      "transformer.h.12.self_attention.dense.bias\n",
      "transformer.h.12.post_attention_layernorm.weight\n",
      "transformer.h.12.post_attention_layernorm.bias\n",
      "transformer.h.12.mlp.dense_h_to_4h.weight\n",
      "transformer.h.12.mlp.dense_h_to_4h.bias\n",
      "transformer.h.12.mlp.dense_4h_to_h.weight\n",
      "transformer.h.12.mlp.dense_4h_to_h.bias\n",
      "transformer.h.13.input_layernorm.weight\n",
      "transformer.h.13.input_layernorm.bias\n",
      "transformer.h.13.self_attention.query_key_value.weight\n",
      "transformer.h.13.self_attention.query_key_value.bias\n",
      "transformer.h.13.self_attention.dense.weight\n",
      "transformer.h.13.self_attention.dense.bias\n",
      "transformer.h.13.post_attention_layernorm.weight\n",
      "transformer.h.13.post_attention_layernorm.bias\n",
      "transformer.h.13.mlp.dense_h_to_4h.weight\n",
      "transformer.h.13.mlp.dense_h_to_4h.bias\n",
      "transformer.h.13.mlp.dense_4h_to_h.weight\n",
      "transformer.h.13.mlp.dense_4h_to_h.bias\n",
      "transformer.h.14.input_layernorm.weight\n",
      "transformer.h.14.input_layernorm.bias\n",
      "transformer.h.14.self_attention.query_key_value.weight\n",
      "transformer.h.14.self_attention.query_key_value.bias\n",
      "transformer.h.14.self_attention.dense.weight\n",
      "transformer.h.14.self_attention.dense.bias\n",
      "transformer.h.14.post_attention_layernorm.weight\n",
      "transformer.h.14.post_attention_layernorm.bias\n",
      "transformer.h.14.mlp.dense_h_to_4h.weight\n",
      "transformer.h.14.mlp.dense_h_to_4h.bias\n",
      "transformer.h.14.mlp.dense_4h_to_h.weight\n",
      "transformer.h.14.mlp.dense_4h_to_h.bias\n",
      "transformer.h.15.input_layernorm.weight\n",
      "transformer.h.15.input_layernorm.bias\n",
      "transformer.h.15.self_attention.query_key_value.weight\n",
      "transformer.h.15.self_attention.query_key_value.bias\n",
      "transformer.h.15.self_attention.dense.weight\n",
      "transformer.h.15.self_attention.dense.bias\n",
      "transformer.h.15.post_attention_layernorm.weight\n",
      "transformer.h.15.post_attention_layernorm.bias\n",
      "transformer.h.15.mlp.dense_h_to_4h.weight\n",
      "transformer.h.15.mlp.dense_h_to_4h.bias\n",
      "transformer.h.15.mlp.dense_4h_to_h.weight\n",
      "transformer.h.15.mlp.dense_4h_to_h.bias\n",
      "transformer.h.16.input_layernorm.weight\n",
      "transformer.h.16.input_layernorm.bias\n",
      "transformer.h.16.self_attention.query_key_value.weight\n",
      "transformer.h.16.self_attention.query_key_value.bias\n",
      "transformer.h.16.self_attention.dense.weight\n",
      "transformer.h.16.self_attention.dense.bias\n",
      "transformer.h.16.post_attention_layernorm.weight\n",
      "transformer.h.16.post_attention_layernorm.bias\n",
      "transformer.h.16.mlp.dense_h_to_4h.weight\n",
      "transformer.h.16.mlp.dense_h_to_4h.bias\n",
      "transformer.h.16.mlp.dense_4h_to_h.weight\n",
      "transformer.h.16.mlp.dense_4h_to_h.bias\n",
      "transformer.h.17.input_layernorm.weight\n",
      "transformer.h.17.input_layernorm.bias\n",
      "transformer.h.17.self_attention.query_key_value.weight\n",
      "transformer.h.17.self_attention.query_key_value.bias\n",
      "transformer.h.17.self_attention.dense.weight\n",
      "transformer.h.17.self_attention.dense.bias\n",
      "transformer.h.17.post_attention_layernorm.weight\n",
      "transformer.h.17.post_attention_layernorm.bias\n",
      "transformer.h.17.mlp.dense_h_to_4h.weight\n",
      "transformer.h.17.mlp.dense_h_to_4h.bias\n",
      "transformer.h.17.mlp.dense_4h_to_h.weight\n",
      "transformer.h.17.mlp.dense_4h_to_h.bias\n",
      "transformer.h.18.input_layernorm.weight\n",
      "transformer.h.18.input_layernorm.bias\n",
      "transformer.h.18.self_attention.query_key_value.weight\n",
      "transformer.h.18.self_attention.query_key_value.bias\n",
      "transformer.h.18.self_attention.dense.weight\n",
      "transformer.h.18.self_attention.dense.bias\n",
      "transformer.h.18.post_attention_layernorm.weight\n",
      "transformer.h.18.post_attention_layernorm.bias\n",
      "transformer.h.18.mlp.dense_h_to_4h.weight\n",
      "transformer.h.18.mlp.dense_h_to_4h.bias\n",
      "transformer.h.18.mlp.dense_4h_to_h.weight\n",
      "transformer.h.18.mlp.dense_4h_to_h.bias\n",
      "transformer.h.19.input_layernorm.weight\n",
      "transformer.h.19.input_layernorm.bias\n",
      "transformer.h.19.self_attention.query_key_value.weight\n",
      "transformer.h.19.self_attention.query_key_value.bias\n",
      "transformer.h.19.self_attention.dense.weight\n",
      "transformer.h.19.self_attention.dense.bias\n",
      "transformer.h.19.post_attention_layernorm.weight\n",
      "transformer.h.19.post_attention_layernorm.bias\n",
      "transformer.h.19.mlp.dense_h_to_4h.weight\n",
      "transformer.h.19.mlp.dense_h_to_4h.bias\n",
      "transformer.h.19.mlp.dense_4h_to_h.weight\n",
      "transformer.h.19.mlp.dense_4h_to_h.bias\n",
      "transformer.h.20.input_layernorm.weight\n",
      "transformer.h.20.input_layernorm.bias\n",
      "transformer.h.20.self_attention.query_key_value.weight\n",
      "transformer.h.20.self_attention.query_key_value.bias\n",
      "transformer.h.20.self_attention.dense.weight\n",
      "transformer.h.20.self_attention.dense.bias\n",
      "transformer.h.20.post_attention_layernorm.weight\n",
      "transformer.h.20.post_attention_layernorm.bias\n",
      "transformer.h.20.mlp.dense_h_to_4h.weight\n",
      "transformer.h.20.mlp.dense_h_to_4h.bias\n",
      "transformer.h.20.mlp.dense_4h_to_h.weight\n",
      "transformer.h.20.mlp.dense_4h_to_h.bias\n",
      "transformer.h.21.input_layernorm.weight\n",
      "transformer.h.21.input_layernorm.bias\n",
      "transformer.h.21.self_attention.query_key_value.weight\n",
      "transformer.h.21.self_attention.query_key_value.bias\n",
      "transformer.h.21.self_attention.dense.weight\n",
      "transformer.h.21.self_attention.dense.bias\n",
      "transformer.h.21.post_attention_layernorm.weight\n",
      "transformer.h.21.post_attention_layernorm.bias\n",
      "transformer.h.21.mlp.dense_h_to_4h.weight\n",
      "transformer.h.21.mlp.dense_h_to_4h.bias\n",
      "transformer.h.21.mlp.dense_4h_to_h.weight\n",
      "transformer.h.21.mlp.dense_4h_to_h.bias\n",
      "transformer.h.22.input_layernorm.weight\n",
      "transformer.h.22.input_layernorm.bias\n",
      "transformer.h.22.self_attention.query_key_value.weight\n",
      "transformer.h.22.self_attention.query_key_value.bias\n",
      "transformer.h.22.self_attention.dense.weight\n",
      "transformer.h.22.self_attention.dense.bias\n",
      "transformer.h.22.post_attention_layernorm.weight\n",
      "transformer.h.22.post_attention_layernorm.bias\n",
      "transformer.h.22.mlp.dense_h_to_4h.weight\n",
      "transformer.h.22.mlp.dense_h_to_4h.bias\n",
      "transformer.h.22.mlp.dense_4h_to_h.weight\n",
      "transformer.h.22.mlp.dense_4h_to_h.bias\n",
      "transformer.h.23.input_layernorm.weight\n",
      "transformer.h.23.input_layernorm.bias\n",
      "transformer.h.23.self_attention.query_key_value.weight\n",
      "transformer.h.23.self_attention.query_key_value.bias\n",
      "transformer.h.23.self_attention.dense.weight\n",
      "transformer.h.23.self_attention.dense.bias\n",
      "transformer.h.23.post_attention_layernorm.weight\n",
      "transformer.h.23.post_attention_layernorm.bias\n",
      "transformer.h.23.mlp.dense_h_to_4h.weight\n",
      "transformer.h.23.mlp.dense_h_to_4h.bias\n",
      "transformer.h.23.mlp.dense_4h_to_h.weight\n",
      "transformer.h.23.mlp.dense_4h_to_h.bias\n",
      "transformer.ln_f.weight\n",
      "transformer.ln_f.bias\n"
     ]
    }
   ],
   "source": [
    "for name, parameter in model.named_parameters():\n",
    "    print(name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lora"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PEFT Step1 配置文件"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LoraConfig(peft_type=<PeftType.LORA: 'LORA'>, auto_mapping=None, base_model_name_or_path=None, revision=None, task_type=<TaskType.CAUSAL_LM: 'CAUSAL_LM'>, inference_mode=False, r=8, target_modules={'.*\\\\.1.*query_key_value'}, lora_alpha=8, lora_dropout=0.0, fan_in_fan_out=False, bias='none', use_rslora=False, modules_to_save=['word_embeddings'], init_lora_weights=True, layers_to_transform=None, layers_pattern=None, rank_pattern={}, alpha_pattern={}, megatron_config=None, megatron_core='megatron.core', loftq_config={}, use_dora=False, layer_replication=None, runtime_config=LoraRuntimeConfig(ephemeral_gpu_offload=False))\n"
     ]
    }
   ],
   "source": [
    "from peft import LoraConfig, TaskType, get_peft_model\n",
    "\n",
    "config = LoraConfig(task_type=TaskType.CAUSAL_LM, target_modules=[\".*\\.1.*query_key_value\"], modules_to_save=[\"word_embeddings\"])\n",
    "print(config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### PEFT Step2 创建模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "Target modules {'.*\\\\.1.*query_key_value'} not found in the base model. Please check the target modules and try again.",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[13], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m model \u001b[38;5;241m=\u001b[39m \u001b[43mget_peft_model\u001b[49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m      2\u001b[0m model\n",
      "File \u001b[1;32md:\\Miniconda\\envs\\geo\\lib\\site-packages\\peft\\mapping.py:183\u001b[0m, in \u001b[0;36mget_peft_model\u001b[1;34m(model, peft_config, adapter_name, mixed, autocast_adapter_dtype, revision)\u001b[0m\n\u001b[0;32m    181\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m peft_config\u001b[38;5;241m.\u001b[39mis_prompt_learning:\n\u001b[0;32m    182\u001b[0m     peft_config \u001b[38;5;241m=\u001b[39m _prepare_prompt_learning_config(peft_config, model_config)\n\u001b[1;32m--> 183\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mMODEL_TYPE_TO_PEFT_MODEL_MAPPING\u001b[49m\u001b[43m[\u001b[49m\u001b[43mpeft_config\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtask_type\u001b[49m\u001b[43m]\u001b[49m\u001b[43m(\u001b[49m\n\u001b[0;32m    184\u001b[0m \u001b[43m    \u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpeft_config\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43madapter_name\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43madapter_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mautocast_adapter_dtype\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mautocast_adapter_dtype\u001b[49m\n\u001b[0;32m    185\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32md:\\Miniconda\\envs\\geo\\lib\\site-packages\\peft\\peft_model.py:1542\u001b[0m, in \u001b[0;36mPeftModelForCausalLM.__init__\u001b[1;34m(self, model, peft_config, adapter_name, **kwargs)\u001b[0m\n\u001b[0;32m   1539\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__init__\u001b[39m(\n\u001b[0;32m   1540\u001b[0m     \u001b[38;5;28mself\u001b[39m, model: torch\u001b[38;5;241m.\u001b[39mnn\u001b[38;5;241m.\u001b[39mModule, peft_config: PeftConfig, adapter_name: \u001b[38;5;28mstr\u001b[39m \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdefault\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs\n\u001b[0;32m   1541\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m-> 1542\u001b[0m     \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[38;5;21;43m__init__\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpeft_config\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43madapter_name\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m   1543\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbase_model_prepare_inputs_for_generation \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbase_model\u001b[38;5;241m.\u001b[39mprepare_inputs_for_generation\n",
      "File \u001b[1;32md:\\Miniconda\\envs\\geo\\lib\\site-packages\\peft\\peft_model.py:155\u001b[0m, in \u001b[0;36mPeftModel.__init__\u001b[1;34m(self, model, peft_config, adapter_name, autocast_adapter_dtype)\u001b[0m\n\u001b[0;32m    153\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_peft_config \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m    154\u001b[0m     \u001b[38;5;28mcls\u001b[39m \u001b[38;5;241m=\u001b[39m PEFT_TYPE_TO_MODEL_MAPPING[peft_config\u001b[38;5;241m.\u001b[39mpeft_type]\n\u001b[1;32m--> 155\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbase_model \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mcls\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m{\u001b[49m\u001b[43madapter_name\u001b[49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mpeft_config\u001b[49m\u001b[43m}\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43madapter_name\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    156\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mset_additional_trainable_modules(peft_config, adapter_name)\n\u001b[0;32m    158\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mhasattr\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mbase_model, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m_cast_adapter_dtype\u001b[39m\u001b[38;5;124m\"\u001b[39m):\n",
      "File \u001b[1;32md:\\Miniconda\\envs\\geo\\lib\\site-packages\\peft\\tuners\\lora\\model.py:139\u001b[0m, in \u001b[0;36mLoraModel.__init__\u001b[1;34m(self, model, config, adapter_name)\u001b[0m\n\u001b[0;32m    138\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m__init__\u001b[39m(\u001b[38;5;28mself\u001b[39m, model, config, adapter_name) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m--> 139\u001b[0m     \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[38;5;21;43m__init__\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43madapter_name\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[1;32md:\\Miniconda\\envs\\geo\\lib\\site-packages\\peft\\tuners\\tuners_utils.py:175\u001b[0m, in \u001b[0;36mBaseTuner.__init__\u001b[1;34m(self, model, peft_config, adapter_name)\u001b[0m\n\u001b[0;32m    173\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_pre_injection_hook(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmodel, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mpeft_config[adapter_name], adapter_name)\n\u001b[0;32m    174\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m peft_config \u001b[38;5;241m!=\u001b[39m PeftType\u001b[38;5;241m.\u001b[39mXLORA \u001b[38;5;129;01mor\u001b[39;00m peft_config[adapter_name] \u001b[38;5;241m!=\u001b[39m PeftType\u001b[38;5;241m.\u001b[39mXLORA:\n\u001b[1;32m--> 175\u001b[0m     \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minject_adapter\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43madapter_name\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m    177\u001b[0m \u001b[38;5;66;03m# Copy the peft_config in the injected model.\u001b[39;00m\n\u001b[0;32m    178\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mmodel\u001b[38;5;241m.\u001b[39mpeft_config \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mpeft_config\n",
      "File \u001b[1;32md:\\Miniconda\\envs\\geo\\lib\\site-packages\\peft\\tuners\\tuners_utils.py:435\u001b[0m, in \u001b[0;36mBaseTuner.inject_adapter\u001b[1;34m(self, model, adapter_name, autocast_adapter_dtype)\u001b[0m\n\u001b[0;32m    433\u001b[0m \u001b[38;5;66;03m# Handle X-LoRA case.\u001b[39;00m\n\u001b[0;32m    434\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m is_target_modules_in_base_model \u001b[38;5;129;01mand\u001b[39;00m \u001b[38;5;28mhasattr\u001b[39m(peft_config, \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtarget_modules\u001b[39m\u001b[38;5;124m\"\u001b[39m):\n\u001b[1;32m--> 435\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[0;32m    436\u001b[0m         \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mTarget modules \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpeft_config\u001b[38;5;241m.\u001b[39mtarget_modules\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m not found in the base model. \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m    437\u001b[0m         \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mPlease check the target modules and try again.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m    438\u001b[0m     )\n\u001b[0;32m    440\u001b[0m \u001b[38;5;66;03m# It's important to set the adapter here (again), because otherwise it can happen that if a 2nd adapter is\u001b[39;00m\n\u001b[0;32m    441\u001b[0m \u001b[38;5;66;03m# added, and it targets different layer(s) than the first adapter (which is active), then those different\u001b[39;00m\n\u001b[0;32m    442\u001b[0m \u001b[38;5;66;03m# layers will be activated, which we don't want.\u001b[39;00m\n\u001b[0;32m    443\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mset_adapter(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mactive_adapters)\n",
      "\u001b[1;31mValueError\u001b[0m: Target modules {'.*\\\\.1.*query_key_value'} not found in the base model. Please check the target modules and try again."
     ]
    },
    {
     "ename": "",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m在当前单元格或上一个单元格中执行代码时 Kernel 崩溃。\n",
      "\u001b[1;31m请查看单元格中的代码，以确定故障的可能原因。\n",
      "\u001b[1;31m单击<a href='https://aka.ms/vscodeJupyterKernelCrash'>此处</a>了解详细信息。\n",
      "\u001b[1;31m有关更多详细信息，请查看 Jupyter <a href='command:jupyter.viewOutput'>log</a>。"
     ]
    }
   ],
   "source": [
    "model = get_peft_model(model, config)\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for name, parameter in model.named_parameters():\n",
    "    print(name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.print_trainable_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step5 配置训练参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "args = TrainingArguments(\n",
    "    output_dir=save_dir,\n",
    "    per_device_train_batch_size=1,\n",
    "    gradient_accumulation_steps=8,\n",
    "    logging_steps=10,\n",
    "    num_train_epochs=5\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step6 创建训练器"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=args,\n",
    "    tokenizer=tokenizer,\n",
    "    train_dataset=tokenized_ds,\n",
    "    data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True),\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step7 模型训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trainer.train()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step8 模型推理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "d:\\Miniconda\\envs\\geo\\lib\\site-packages\\numpy\\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:\n",
      "d:\\Miniconda\\envs\\geo\\lib\\site-packages\\numpy\\.libs\\libopenblas.FB5AE2TYXYH2IJRDKGDGQ3XBKLKTF43H.gfortran-win_amd64.dll\n",
      "d:\\Miniconda\\envs\\geo\\lib\\site-packages\\numpy\\.libs\\libopenblas64__v0.3.21-gcc_10_3_0.dll\n",
      "  warnings.warn(\"loaded more than 1 DLL from .libs:\"\n",
      "Could not find the bitsandbytes CUDA binary at WindowsPath('d:/Miniconda/envs/geo/lib/site-packages/bitsandbytes/libbitsandbytes_cuda116.dll')\n",
      "The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Human: 考试有哪些技巧？\n",
      "\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: 考试有哪些技巧？\n",
      "Assistant: \n"
     ]
    }
   ],
   "source": [
    "from peft import PeftModel\n",
    "from transformers import AutoModelForCausalLM, AutoTokenizer\n",
    "import os\n",
    "data_dir = '/data/datasets/alpaca_data_zh/alpaca_gpt4_data_zh.json'\n",
    "pretrain_model_dir = \"/data/models/huggingface/bloom-1b4-zh\"\n",
    "save_dir = '/data/logs/conversation_rebot_for_1b4_lora_tuning'\n",
    "# 在一个jupyter文件中，如果前面已经加载了模型，并对模型做了一定修改，则需要重新加载原始模型\n",
    "model = AutoModelForCausalLM.from_pretrained(pretrain_model_dir)\n",
    "tokenizer = AutoTokenizer.from_pretrained(pretrain_model_dir)\n",
    "peft_model = PeftModel.from_pretrained(model=model, model_id=os.path.join(save_dir,\"checkpoint-1220/\"))\n",
    "peft_model = peft_model.cuda()\n",
    "ipt = tokenizer(\"Human: {}\\n{}\".format(\"考试有哪些技巧？\", \"\").strip() + \"\\n\\nAssistant: \", return_tensors=\"pt\").to(peft_model.device)\n",
    "print(tokenizer.decode(peft_model.generate(**ipt, max_length=256, do_sample=False)[0], skip_special_tokens=True))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "transformers",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.19"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
